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(57) Abstract: 

PURPOSE: To improve the sharpness of an image by 
applying adaptive interpolation to a digitized image 
signal obtained from an image sensor. 

CONSTITUTION: An image sensor 12 is covered by a 
color filter array 13 and each photo site of the image 
sensor 12 produces a single color. An analog image 
signal generated by the image sensor 12 is digitized and 
stored in an image buffer 18. The signal stored in the 
image buffer 18 is fed to a digital signal processor 22, 
where interpolation processing is conducted. In the 
interpolation processing, additional luminance values 
with respect to a pixel whose luminance values (green) 
are lost are obtained from luminance values of adjacent 
pixels of the concerned pixel through interpolation. In 
this case, a gradation value is obtained from a 
difference of the luminance values in a horizontal or 
vertical direction and the gradation value is compared 
with a threshold value. Then, the direction 
(combination) of pixels used for the interpolation is 
adaptively decided based on the comparison. 
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(57) [Sj»] 

Hi#* l &<d&.#7— m&£.l&tZ>o 

i-5Wi0W*»*ffiS:, ^<DiIi&wj£»<E>Bi3S<z>Wa^i: 
©»«ll[oSI*»e>|«il«[Sr*«>, io?fif»MI[«rL.*v^ffi 
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SfrfBfV t^/Wfc>f ^{f-g-S:, 7* mm 

mm hffi«{^*i-rs{+jpw»«ffisr, -?r^^^ mm 10 
y\iLW<vi&m(D7* mm bfm^»««*» 

>f f+JPW»««S:*W'*-Sfc»0»a 20 

B9iM>* <H2 oco>r ^ — versus, Tk^injx^s 
izmm 3 ] m^m i im^skki^-c, 

[0001] 

[0 0 0 2] 

• y-?^ — >- : regular pattern ) i^^f^*t"5Hj^{iEi^ 
(CFA) Ea^^-^MoT, &telTZ>\??±/l>{iL 

3fc<£>:7/]^7 7 — f — 50 




%fMW-7 - 5 9 0 9 8 
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**-rs tub {^mm^^m t * s 0 
[ooo3]-«tti*^7^fAij:^^ ^y- 

tfefttfj^l^^y >-y$tlfz. (sparse 
ly sampled) ^77 — t ( "tr^^i M 1 fe) 

l^^K, V. 7;^- (RGB) <D7?Vjj=7 — 

v> ^ t°^-tr/vrTi:^RGB) <Z>B«M 

l^ty/!)y^§tl5W (high-frequency) ff-S§-tfS 

/!//!) ^ -^/u-ySr4*t5 

ibtfVM y = T (bilinear : ffifBl&m^bfr 

Lfc^y— V (missing green ) J ffiSr*SKM-5rt:«> 

y— V^c^j fc^ir/w) o^ffi^cofflflo^y — 

4, 6 4 2, 6 7 8^1|W*£;h/ri^5o 
[0 0 0 4] 

-7^77^ h (artifact) j&S^CS r £ *T?S>5 0 
B9JBtt, Wx.«*H1«fM : »4, 6 3 0, 3 0 7^»T 
$ ti6 J: 5 * «fc 9 m^^Wf^W^ffl^^ ^ J: 19 
^L-CT^^o £<Z>4, 6 3 0, 3 0 7^H#fFT^i, S 
(neiborhood) (7^-^-^) (d 

/^=^yXi^S:fflv^T>:^IE■7 ^ — ^^SWfigStbSo 
te\ r ^ 4 , 6 3 0, 3 0 7 ^*H#fflFT?te\ ^ V 

■feyv^ r — * = ^ t:° - ^ (niSfflE $ iMtrr V X h t 
[0 0 0 5] tW««LfcJ:5i^ ±3Bt**^ffi^<t 
7^ hdS38^i-Sr i:"e*>S 0 _hlS4, 6 3 0, 3 0 7 

[0 0 0 6] *|8M<z>lWtt, ^F^{^^^ h£ri#;*:£it 
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[0 0 0 7] ££>1^ ffl^l^ft $7—=zy*J<D 

x) %xmz£yffim*ff?z.tx*hz> 0 

[0 0 0 8] 

v^-fe^^ft^^^— • ^ MM* h (phot 10 
osite ) ^tL, ^ixfeSScO)*^ — • MM MS 
fllS:^rit-t-SdS % MM h<7>fcgi- 

4ol^Tf3\ fefc* 1 oo* 7-fi^*S:4fiRt 5o 

-i X : image-wise) ^ (tifdL,^ ^<Ds<& — l/\z. 

*3lvr. ^<oj0>cd:7;*- MM hffiK^WSfiSSrfiF^* 

tuT»8rrs«#*w*ai:, b) nfriBm^wn 

*BW*«*ffiS:, -^co^*- MM hffi«<£>ifi»<Z>:7;*- K- 20 

im h n % mmmfa h mm-t sr^aoti 

[0 0 0 9] Sbl^, HtflS^n-fe?/^- 
te, a) t t2 0^p«— ^|pI^*3^S»«fii[ 

ra^J;!>fi»ii (/7fV^Vb • ^y^— : grad 
ient values ) & £.f£1r & k , b) HUfEISIHffifiSr 
£ t lo©l,g:i^£Jfc«rr5*&£:, c ) buSB 

* < t fc 2o<7M ^ — v^fpjeD 5 t> 1 o£rigtR*f-£^© 30 
d) »E»ii**rft^-S^SJ:5»«*nfc** 

£, 

[0 0 10] #^<z>£fca*fci\ fea^n-fe^Srig* 
[0011] 

[0 0 12] £t\ 0iXtfl2 2i£o^Tlft|IH-5o a 
^W^A- • # ^ ? tta#A^SB 2 i: ttffl . l£^g|5 4 
^tt^H^o A*SfB2fi, (f^Xv^^ h : subjec 

t) (ig^it-r) frh<D^ * — v^^m ^-^tyti 
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MM h^2RS7WS:ttftyiM 2 JS, J^£p<7> 

(CCD) T*fc6 0 irVi^l 2 ft, s<4*Y— • 
(Bayer array) £ ^ 7 "7 7 W 

(CFA) 1 3id£oT»b;h/0>£ 0 r(7VM^ — • 
7Wf:o^TH *B#ff!g3, 9 7 1, 0 6 5^1: 

XO*X/w- ) a*, ^rcy^;— K««{^iaaStufe» 
^ra^iS:ffibTV^S<D^«we 

3 V : charge information) <D7 & MM M^:Jo 

^T£j$£*v£o r<7>S^«««, UBMY^K14 
^#t»Six, tU^^^- Kl 4l*£<0«Wtt«8:«-* 

(7)T^n ^ ^ — t^fS ^(iA/D ^ 1 6 l^mS 

rrr^iirat67tn/A^(f^^f^ 

AM) T*fc<5o 
[0 0 13] >^i3-fe^1^2 Ote, (K3te8fBl 0(D^<9^ 

m^-ir^V^hi^^L-. ^^-^y7r 1 8£*SjL 
TA/D=t>-^— ^ 1 6 ^rf^ffij^iir^r 

^^A^a5 2^^jffi)i~^ mmy p v*tyy-2 o 

/urfy XjU^^-x., fflPfl$axfc«-g-S:=i^^^2 6*^ 

[0014] *ra, xr^rtb^BBii-rsfflattiiw** 
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[0 0 15] A^£P2f^ mig&X^ y<o^^<^—>a 

>\^*)&ou~v mm -cf^Bi-So 

i:t, Htfii o o o^i»^b*»n*-c^Jwm, 

— S?»ffiffcJ\ ir^iM 2^^^- MM h^fetB^ffi* 

gf*p h ft, MfLvi^ WM<z> \?*f*7 ^ —As K u— V 

2 0£»£>i?^lM 2. A/D=a^^— ? 1 6, ;y 
1 8^ffe|&$tL5B»«^co*9 3SLU- Hi, -b 

[0 0 16] ^(7)M(a5lo^U^HT 

^Scofe^tc:il^"r^ r t 5ri ^sfetf fettSo 
iEr«9, hi?y7^ - h<Dm<D^ 

^•*^7tt, iftK-rs— i&<o>r*—i ^i-^LTig 

fe6 0 ^COf-fc, HI 1 s^^J/^y^T 1 8 

[0 0 17] «ff«*3§3 0tt7 p C2-fes/-^2 Ofc:g&R£ 
3t) . 7>f^K/7U-A, {g/^^y (^yf!lfi 
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y«r»*^T?*5!9. rW\ *«Sttfc^^-^» 

[0 0 18] fV^^^7 P Pt 5 /t2 2fl 

y ^l — : missing data values ) £>ttHffltt:EI 2 £th 
y— >-^C^ (missing green ) J CO t 0 ^ -fevu (0$?, 

(high frequency information) N 9* V —^CO 

is 3 >- : rendition ) dSfifeilSixSo #C^^ fe^l*^ 

*^>r y =-T*tet^«tsttraf-<t i9*«>e>*b, cfa^ 
TK^r^R^iiL^r^coffl^co^y-vbi^ir/Hi 

dient ) *5p*|n]^t, ^tg^f^^t. *fc 

[0 0 19] ^(DX fflF^^air^co^l c/)^^-/ 

30 ^niy^4 0|:$Stt5J:5^ W<H2o 

6) , ^n^3 8t^*5VNTfeH^J(|LaStt5o Sf^ 
50 ^T3^^ 4 8tc*3V^T, 2#:7C^-f y ^TWFplSrffl 
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(RGB) ^SWfiELTfc ctv^U, *fcf*fe§g 
t Ltfttt L £ <b IdMfiffiSrtTo T t> £ v \ 
[0 0 2 0] Xy — ^^^Lfct^i^WSB 

(MTF) ^^r-r^irV1^i^i^CTpgi5i~^r.ir^-e^ 
[0 0 2 1] rtb&^PgW^SflBl^^tJft^SttS^/ 

-^f-^^4 0^^f^y —tel^TCDigl *) X*fo&o 
[0 0 2 2] A. (Ttc^PPgP > L^Wfi) ;&>o (fi 

/!✓ • -r* — jV : high scene spatial detail ) £r^l 20 

K/co/cffiig (^h^^^-T : structure ) J^ftV^c: 
[0 0 2 3] B. (TK^PPgf^ ^ L#V^) j&>o (S 

/h" fc<z>-<?&i^-e, shrews x^*) ^ 

[0024] c. (i&wmm > u^vMe) a>o ® 
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[0 0 2 5] D. (yk^mm ^ LtvMfi) ^ (H 

[0 0 2 6] W^CFA^^ — ^{CRR^Stt5fc^"e 
5> ftRGB^M^ — (Bayer) '*7^7>f;^7W 

tt\ RGB^Ti(^tJ: 5ftW»ftft'Wir— ■ X* 

[0 0 2 7] 

G R 
B G 

[0 0 2 8] IU4 (A) ~ (D) {CfcJu iWSWlo 
(£>t 0 ^i?/lx (A"ell/u-e*>S) MUTT 5 4 0(^)7x- X 

(-«t tJBSSUStbSo ^^-Xd^T, G (x, 

y) , r (x, y) , b (x, y) th^mm^ mm 
(x) RTjmmjjfo^& (y) i^o^r-r ^-^^f- 

l£j:oT8!^Lfc«-CS>5o Adap G(X,Y) J^TcOT 

/w=ry x^&jj^TMIHSfrfc^y -^«T-fc5 0 

[0 0 2 9] 
[»2l 



#M¥7 - 5 9 0 9 8 



9 10 

For Gdiff-hor = |G(-1,0) - 0(1,0)1 

Gdiff-ver = |G(0,-1) - G<0,1)| 

Threshold = Predetermined value 
If (Gdiff-hor < Threshold) and (Gdiff-ver < Threshold) 
Or (Gdiff-hor > Threshold) and (Gdiff-ver > Threshold) 

Then lAdap G(X,Y) = [G<-1,0) + G<1,0) + G(0,-1) + 
G(0,1)]/4I 

If (Gdiff-hor < Threshold) and (Gdiff-ver > Threshold) 
Then lAdap G(X,Y) [G(-1,0) + G(1,0)]/2I 

if (Gdiff-hor > Threshold) and (Gdiff-ver < Threshold) 
Then lAdap G(X,Y) = [G(0,-1) + G(0 r l)]/2| 



fz.tdl,~^X\ "Threshhold" rb^VMttj £r N "Pr 
edetermined value " ITJfjgfllj ^r^-t* 0 
[0 0 3 0] r©7;P^yXAT*fi % tf, Gdiff-hor 

V— VffiC^IS (<£>*6*M£) £}£«§U Gdiff-ver frmZ 

fit) ir^ttLTV^o %z Lt, Gdiff-hor Sl^Gdiff-ve 

(^L^VMBJ: >9/Jn^V^^^ Adap G(X,Y) ^4o^)t° 
^Hr/W^y—:^*^ 5 !^ Gdiff-hor 2$ 



* L^VvfgJ; 9/h£< n Gdiff-ver 3&SL#l/Mft <fc 0:*c#^ 
Adap G(X,Y) ^7k^^(D^ V —^i^(D^^I t 
LTV^§ 0 Gdiff-hor J&3 V Mtt <fc 9 < * Gd 

20 iff-ver tf* L£ Wfit X V) /h £ V Adap G(X,Y) £: 

[0 0 3 1 ] H4 (A) — (D) 604o©7x-X|:S 

[0 0 3 2] 
[*3] 
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7x-X 0 0 (134 (A) ) 
G = G{0,0) 

B = 0.5 * [B(0,-1) - Adap 0(0,-1) 4- B(0,1) - Adap 

G(0, 1) ] + G(0,0) 

R = 0.5 * [R(-1,0) - Adap G(~1,0) + R(1,0) - Adap 

G(1,0)J + 0(0,0) 



y*-x 0 1 (04 (B) ) 

R = R(0,0) 

G = Adap G ( 0 , 0 ) 

B = 0.25 * [B(-l,-l) - Adap G(-l,-l> + B(-l,l> - 
Adap G(-l,l) + B<1,-1) - Adap G(l,-1) + 
B(1,1J - Adap G<1,1)] + Adap G(0,0) 

7~-X 1 0 (M4 (C) ) 

B = 3(0,0) 

G = Adap G{0,0) 

R = 0.25 * [R<-1,-1) - Adap G{-1,-1) + R(-l,l) - 
Adap G<~1,1) + R(l,-1) - Adap G(l,-1) + 
R(l,l) - Adap G(l,l)] -*- Adap G(0,0) 

1 1 (04 (D) ) 

G = G(0, 0) 

R = 0.5 * [R(0,-1) - Adap G<0,-1) + R(0 # 1) - Adap 

G (0 , 1) ] + Adap G < 0 , 0 ) 
B 0.5 * [B(-1,0) - Adap G(-1,0) + B(1,0) - Adap 

G(l, 0) ] + Adap G(0, 0) 



— ^as^OSLTV^;^— X (7x^X0 lAXfl 0) 
i£*3^T, (%-?m/vtd) t^ir/W^L 

-g, b-g) y -T^mmzfoZo & 

[0 0 3 3] *%i^Mft»ilT-^7^ 7(^*5V>tffl 



£\ El 2 <£>*8f!a&ffite ^>t°^~?co7 p v ^7^i^£^ 
[0 0 3 4] r^l-Cfi, »ia*3Ufi«ia:«F^#fi8UT* 

[0 0 3 5] 

* mm h\zttj&T&&K°?±su<D&%ffimi,T7/i'X 
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[0 3 1 • S^*-* h y 

y (7? 4 o (Z) 7 ^ — x $r m 1~ in -e & £ 0 
i o h*sb 
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* l 2 >f^— ^tyt 

13 13=7 — 7 4^97^4 (CFA) 

1 6 A/TD^l//*— 9 

1 8 <>{ *—i?s<y7T 

2 0 y^n-feyi*- 

2 2 i;^;^>/t;^P t y t 

2 4 ^^ey^-K 
26 9 

2 8 ^n-fe^^s^r 

3 o m¥m^ 
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